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DETAILED ACTION 



The following action is in response to the Request for Continued Examination (RCE) 
filed under 37 CFR 1 .53(d) for the instant application on May 21 , 2009. Applicants 
have properly set forth the RCE, which has been entered into the application. 
Accordingly, the amendment submitted May 21, 2009, has been entered and an 
examination on the merits follows herewith. Claims 1 , 24-26, 30-31 , 34 and 38-40 
have been directly amended. Claim 41 has been newly added. Claim 18 has been 
canceled. Claims 1-17 and 19-41 are pending and have been considered below. 



Status of Claims 

Claims 1-17 and 19-40 are pending in the case. Claims 1, 24, 25, 26, 30, 31, 34, 38, 
39 and 40 are the independent Claims. 

Claims 1-17 and 19-40 remain rejected under 35 U.S.C. 103(a). 



Claim Rejections - 35 (JSC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claims 1-13, 15-17 and 19-41 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over R. Douglas Riecken, "Adaptive Direct Manipulation", IEEE, 1991, Vol. 
2, pages 1115-11 20, previously presented as "Riecken", in view of Nelson (US PGPUB 
2004/0008635), previously presented as "Nelson". 

Regarding Claim 1 , Riecken discloses the claimed aspect of a method for displaying 
a representation of content (page 1115 paragraph 3: "a prototyped GUI system which is 
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adaptive to user performance"), comprising: monitoring user behavior (pages 1119 
paragraph 1 : "While monitoring user interactions") while interacting with a first 
representation of content (pages 1116 paragraph 2: "when a system initially presents its 
interactive graphical objects"; the initial presentation is a first representation); 
determining interaction information from the user behavior (page 1119 paragraph 1 : "KS 
applies statistical inference to determine if modifications in user behavior have 
occurred"); maintaining the interaction information (page 1119 paragraph 2: "KS 
computers and logs the button dialogue behavior"); and deforming a second 
representation of content using the interaction information (page 1119 paragraph 5: 
"then all the KSs will collaborate to construct a new layout for the button matrix"); and 
displaying the second representation of content (page 1119 paragraph 5: "then all the 
KSs will collaborate to construct a new layout for the button matrix", page 1118 
paragraph 2: "The display manager KS manages and posts to the blackboard all 
graphical events presented to the user"), wherein a first display condition of the first 
display interaction area where the first representation is displayed is different from a 
second display condition of a second display area where the second representation is 
displayed (page 1119 paragraph 5: "If the error rate (defined by the spatial variance KS) 
of button selections increases above a system defined threshold (current value is 3%), 
then all the KSs will collaborate to construct a new layout for the button matrix."). 

Riecken further discloses that the user behaving includes interacting with a first 
display interaction area and at least a second display interaction area (page 1119 
paragraph 1 : "tracks a user's performance based on the frequency and recency of finger 
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selections within a given area", paragraph 3: "general changes in the physical location 
within each button area where the user is touching", paragraph 4: "a set of buttons 
touched by the user", paragraph 5: "a spectrum ranging from touches which are located 
in the center of a button to touches which completely miss a desired button"). While 
Riecken does not explicitly disclose the interaction information including an explicit 
order in which the first and second area are selected, Riecken does disclose that the 
recency of the interactions are recorded (page 1119 paragraph 2: "both the frequency 
and recency of user button dialog." Recency is understood to be information about 
which interactions are most 'recent', i.e. an order to the interactions). Therefore it would 
have been obvious to one having ordinary skill in the art and the teachings of Riecken 
before them at the time the present invention was made to store the order of interaction, 
or recency as suggested by Riecken, for the various interaction areas as part of the 
user behavior model taught by Riecken. One would have been motivated to do this in 
order to better predict the accuracy of user interaction, and deform the buttons based on 
such, as suggested by Riecken (page 1119 paragraphs 1-6). 

While Riecken does not teach explicitly the claimed aspect of digital content, 
Riecken does disclose graphical objects (page 1118 paragraph 2: "each active 
graphical object"). Applicant should duly note that graphical objects could be digital 
content. Even if not, Nelson discloses the claimed aspect of digital content in a multi- 
participant conference system with controllable content. (Nelson, Abstract, FIG. 14 
shows mixed modes of digital content displayed concurrently). It would be obvious to 
one of ordinary skill in the art at the time of the invention to expand Riecken's layout 
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based on user behavior concept to include Nelson's digital content. One would have 
been motivated to make this expansion to allow the collaborative blackboard of Riecken 
to include forms of content other than graphical objects, providing for a richer user 
experience, as suggested by Nelson ([0011]). 

Regarding Claim 2, most of the limitations have been met in the rejection of Claim 1 . 
See details for Claim 1 rejection. Riecken further discloses wherein deforming a second 
representation includes deforming an active area of the second representation (page 
1118 paragraph 2: "each active graphical object", page 1117 paragraph 1 : "have their 
color changed"). 

Regarding Claim 3, most of the limitations have been met in the rejection of Claim 1. See 
details for Claim 1 rejection. Riecken further discloses wherein deforming a second 
representation includes deforming a layout of the second representation (page 1117 paragraph 2: 
"their horizontal spacing may vary slightly"). 

Regarding Claim 4, most of the limitations have been met in the rejection of Claim 1 . 
See details for Claim 1 rejection. Riecken further discloses the claimed aspect of the 
first and second representation are of the same content (page 1 1 17 paragraph 2; the 
button matrix changes in appearance, but the content delivered, that of the buttons, 
does not change, i.e. remains the same). 
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Regarding Claim 5, most of the limitations have been met in the rejection of Claim 4. See 
details for Claim 4 rejection. Nelson further discloses the claimed aspect of the digital content is 
at least one of a web page, a digital document, a digital image, an electronic book, a digital slide, 
and a graphical user interface, wherein a graphical user interface and plurality of participants 
(audio/video stream for each participant) conferencing environment are provided (Abstract, 
[0003], [0005]). 

Regarding Claim 6, most of the limitations have been met in the rejection of Claim 1. See 
details for Claim 1 rejection. Riecken further discloses the claimed aspect of the second 
representation is scaled in relation to the first representation (page 1117 paragraph 2: "but their 
horizontal spacing may vary slightly."). 

Regarding Claims 7 and 8, most of the limitations have been met in the rejection of Claim 1 . 
See details for Claim 1 rejection. Nelson discloses the claimed aspect of the first representation is 
a representation of first digital content and the second representation is a representation of 
second digital content and first representation is a representation of a first graphical user 
interface and the second representation is a representation of a second graphical user interface 
(Nelson Fig. 8: different participants view different layout of the graphical representation with 
digital content). 



Application/Control Number: 10/672,980 
Art Unit: 2175 



Page 7 



Regarding Claims 9, most of the limitations have been met in the rejection of Claim 
1 . See details for Claim 1 rejection. Riecken discloses the claimed aspect of monitoring 
user behavior while interacting with the first representation comprises: monitoring user 
interaction with the first representation of digital content (page 1119 paragraph 1 : "While 
monitoring user interactions"); and determining interaction areas from the user 
interaction with the first representation (page 1119 paragraph 1 : "within a given area"). 

Regarding Claim 10, most of the limitations have been met in the rejection of Claim 
9. See details for Claim 9 rejection. Riecken further discloses the claimed aspect of 
evaluating user interaction with the interaction areas (page 1116 paragraph 6: "buttons 
are selected by a user to initiate system actions"). 

Regarding Claim 1 1 , most of the limitations have been met in the rejection of Claim 
1 . See details for Claim 1 rejection. The rejection for Claim 5 applies to Claim 1 1 . See 
rejection details for Claim 5. 

Regarding Claim 12, most of the limitations have been met in the rejection of Claim 
1 . See details for Claim 1 rejection. Riecken further discloses the claimed aspect of 
maintaining the interaction information includes maintaining the interaction information 
with an identification of the content from which the interaction information was 
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determined (page 1118 paragraph 2: "a graphical object identifier (name)", page 1119 
paragraph 2: 'KS computes and logs the button dialogue behavior of the user"). 

Regarding Claim 13, most of the limitations have been met in the rejection of Claim 
9. See details for Claim 9 rejection. Riecken further discloses the claimed aspect of 
deforming the second representation comprises: determining interaction areas of the 
second representation corresponding to the first representation; deforming the 
corresponding interaction areas (page 1117 paragraph 2: color changes for the 
corresponding interaction areas). 

Regarding Claim 15, most of the limitations have been met in the rejection of Claim 
1 . See details for Claim 1 rejection. Nelson further discloses the claimed aspect wherein 
deforming the second representation includes applying an animation to areas of the 
second representation using the interaction information (Abstract, FIG. 7, FIG. 8: video 
content is displayed as one of a multitude of digital contents). 

Regarding Claim 16, most of the limitations have been met in the rejection of Claim 
1 . See details for Claim 1 rejection. Riecken further discloses the claimed aspect of 
determining interaction information from the user behavior includes determining a 
degree of interaction with at least one area of the first representation (page 1119 
paragraph 2: "frequency and recency of user button dialogue"). 
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Regarding Claim 17, most of the limitations have been met in the rejection of Claim 1 . See 
details for Claim 1 rejection. Riecken further discloses the claimed aspect of determining 
interaction information from the user behavior includes determining a sequence of interaction 
with the first representation (page 1119 paragraph 2: "frequency and recency of user button 
dialogue"). 

Regarding Claim 19, most of the limitations have been met in the rejection of Claim 1. See 
details for Claim 1 rejection. Riecken does not specifically teach the claimed aspect of first 
representation is not deformed when deforming the second representation. However, Nelson 
discloses the claimed aspect of different layouts for each participant, it would be obvious to one 
of ordinary skill in the art at the time of the to keep the original version of the graphical user 
interface as well, because this would allow the user to go back to the original default layout. 

Regarding Claim 20, most of the limitations have been met in the rejection of Claim 1. See 
details for Claim 1 rejection. Riecken further discloses the claimed aspect of the first and second 
representation are representations of the same content, and wherein: the second representation is 
deformed without modifying the content (page 1117 paragraph 2; the button matrix changes in 
appearance, but the content delivered, that of the buttons, does not change, i.e. remains the 
same). 
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Regarding Claim 21, most of the limitations have been met in the rejection of Claim 1. See 
details for Claim 1 rejection. Riecken discloses the claimed aspect of maintaining the interaction 
information comprises storing the interaction information at at least one of a client-side device, a 
server, and a proxy server (page 1119 paragraph 2: "spatial variance KS computes and logs", 
page 1117 paragraph 3: "the ADM system is designed and implemented using AI blackboard 
technologies [..] provided by a set of blackboard knowledge sources (KS)", page 1 116 paragraph 
5: "In designing and implementing the ADM system [..] a physical interface based on touch 
screen technologies was developed"; client device). 

Regarding Claim 22, most of the limitations have been met in the rejection of Claims 4 and 
1. See details for Claims 4 and 1 rejection. Riecken further discloses the claimed aspect of 
maintaining the interaction information comprises: adding the interaction information to a file 
containing data for the content (page 1119 paragraph 2: "KS computes and logs the button 
dialogue behavior", page 1119 paragraph 3: "descriptive features of variance [..] include: general 
changes in the physical location [..] geometry defining the size and shape [..] spatial relationship 
between a set of buttons [..] qualification and quantification of touches to the button", page 1118 
paragraph 2: "a graphical object identifier"). 

Regarding Claim 23, most of the limitations have been met in the rejection of Claim 1 . See 
details for Claim 1 rejection. Riecken further discloses the claimed aspect of monitoring a user's 
behavior including highlighting a textual passage of content or selecting a portion of an image or 
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document (page 1119 paragraph 3: "general changes in the physical location (cluster of pixel 
points) within each button area where the user is touching"; selection of a portion of an image). 
Nelson further discloses the claimed aspect of monitoring user behavior while interacting with a 
first representation of digital content comprises monitoring a first user's behavior while 
interacting with the first representation; and deforming a second representation of digital content 
using the interaction information comprises deforming a second representation presented to a 
second user (FIG. 15: wherein user activity is monitored 224, FIG. 8: wherein different interface 
layout is illustrated for each user). 

Regarding Claim 24, most of the limitations have been met in the rejection of Claim 1. See 
rejection details for Claim 1. Riecken further discloses wherein user behavior includes how often 
or how many times an area is interacted with or in what order a user interacts with the area (page 
1117 paragraph 2: "based on the frequency in which a user touches a 'user preferred' target 
location within the buttons", page 1119 paragraph 2: "both the frequency and recency of user 
button dialogue"). 



Regarding Claim 25, the rejection for Claims 1, 2 and 9 apply to Claim 25. See rejection 
details for Claims 1, 2 and 9. 
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Regarding Claim 26, the rejection for Claims 1 and 9 apply to Claim 26. See rejection details 
for Claims 1 and 9. Riecken further discloses identifying an interaction area determined to be of 
interest using observed user interaction which includes how often or how many times an 
interaction area is selected (page 1117 paragraph 2: "based on the frequency in which a user 
touches a 'user preferred' target location within the buttons"). 

Regarding Claim 27, most of the limitations have been met in the rejection of Claim 26. See 
details for Claim 26 rejection. The rejection of claim 9 applies to Claim 27. See rejection details 
for Claim 9. 

Regarding Claim 28, most of the limitations have been met in the rejection of Claim 26. See 
details for Claim 26 rejection. The rejection for Claim 1 applies to Claim 28. See rejection details 
for Claim 1 . 

Regarding Claim 29, most of the limitations have been met in the rejection of Claim 28. See 
details for Claim 28 rejection. The rejection for Claim 5 applies to Claim 29. See rejection details 
for Claim 5 . 
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Regarding Claim 30, most of the limitations have been met in the rejection of Claim 1 . See 
details for Claim 1 rejection. Riecken further discloses that the user behavior includes how often 
or how many times an area of the first representation is interacted with or in what order a user 
interacts with the area (page 1117 paragraph 2: "Performance evaluation (by which the system 
functions) is based on the frequency in which a user touches a 'user preferred' target location 
within the buttons with specific fingers (excluding the thumb)."). 

Regarding Claim 31, the rejection for Claims 1, 2, 3, and 1 1 apply to Claim 31. See rejection 
details for Claims 1, 2, 3 and 11. 

Regarding Claim 32, most of the limitations have been met in the rejection of Claim 31. See 
details for Claim 31 rejection. The rejection for Claims 1 and 13 apply to Claim 32. See rejection 
details for Claims 1 and 13. 

Regarding Claim 33, most of the limitations have been met in the rejection of Claim 31. See 
details for Claim 31 rejection. The rejection for Claim 5 applies to Claim 33. See rejection details 
for Claim 5 . 
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Regarding Claim 34, the rejection for Claims 1, 2 and 3 apply to Claim 34. See rejection 
details for Claims 1, 2 and 3. 

Regarding Claims 35 and 36, most of the limitations have been met in the rejection of Claim 
34. See details for Claim 34 rejection. The rejection for Claims 12 and 13 apply to Claim 35. See 
rejection details for Claims 12 and 13. 

Regarding Claim 37, most of the limitations have been met in the rejection of Claim 36. See 
details for Claim 36 rejection. The rejection for Claim 20 applies to Claim 37. See rejection 
details for Claim 20. 

Regarding Claim 38, the rejection for Claims 1, 2 and 3 apply to Claim 38. See rejection 
details for Claims 1, 2 and 3. 

Regarding Claim 39, the rejection for Claims 1, 2 and 3 apply to Claim 39. See rejection 
details for Claims 1, 2 and 3. 



Regarding Claim 40, the rejection for Claims 1, 2 and 3 apply to Claim 40. See rejection 
details for Claims 1, 2 and 3. 
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Regarding Claim 41, most of the limitations have been met in the rejection of Claim 1. See 
details for Claim 1 rejection. Riecken further discloses wherein deforming a second 
representation includes deforming an active area of the second representation corresponding to 
the first display interaction area and at least the second display interaction area of the first 
representation (page 1118 paragraph 2: "each active graphical object", page 1117 paragraph 1 : 
"have their color changed"). 

Claims 14 is rejected under 35 U.S.C. 103(a) as being unpatentable over Riecken in view of 
Nelson and in further view of Robertson, "The Task Gallery: A 3 D Window Manager", CHI 
2000, April 1-6 2000, previously presented as "Robertson". 

Regarding Claim 14, most of the limitations have been met in the rejection of Claim 13. See 
details for Claim 13 rejection. Riecken does not disclose the claimed aspect of deforming the 
corresponding interaction areas includes at least one of enlarging the interaction areas, applying a 
fisheye perspective to the interaction areas, and zooming the interaction areas. Nelson discloses 
the claimed aspect of zooming the interaction areas (FIG. 13). Riecken and Nelson do not teach 
the claimed aspect of fisheye effect. Robertson discloses the claimed aspect of a fish-eye effect 
(FIG. 1 : wherein fisheye effect is illustrated). It would be obvious to one of ordinary skill in the 
art at the time of the invention to simply substitute the fisheye effect of Robertson for the 
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zooming effect of Riecken's layout manipulation and Nelson's digital content delivery system, 
yielding the predictable result of applying a fisheye perspective to interaction areas. 

Response to Arguments 

Applicant's arguments filed May 21 , 2009, have been fully considered but they 
are not persuasive. 

Applicant alleges, with regards to claims 1, 24-26, 30-31, 34, and 38-40, that 
neither Riecken nor Nelson teach or suggest the amended limitations of the above 
claims. The Examiner respectfully disagrees and directs Applicant's attention to the 
corresponding rejections above. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to ANDREW TANK whose telephone number is (571)270- 
1692. The examiner can normally be reached on Mon-Thur 9am-5pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, William Bashore can be reached on (571)272-4088. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/A. 1.1 

Examiner, Art Unit 2175 
July 27, 2009 

/William L. Bashore/ 

Supervisory Patent Examiner, Art Unit 2175 



